The structure of ordinary: Hui vernacular settlements and architecture in China

Ponència presentada a: Session 8: Dimensiones psicosociales de la arquitectura y el urbanismo / Psycological dimensions of architecture and plannin

A Note on the Maximum Genus of Graphs with Diameter 4

Let G be a simple graph with diameter four, if G does not contain complete subgraph K3 of order three

IsoBN: Fine-Tuning BERT with Isotropic Batch Normalization

Fine-tuning pre-trained language models (PTLMs), such as BERT and its better variant RoBERTa, has been a common practice for advancing performance in natural language understanding (NLU) tasks. Recent advance in representation learning shows that isotropic (i.e., unit-variance and uncorrelated) embeddings can significantly improve performance on downstream tasks with faster convergence and better generalization. The isotropy of the pre-trained embeddings in PTLMs, however, is relatively under-explored. In this paper, we analyze the isotropy of the pre-trained [CLS] embeddings of PTLMs with straightforward visualization, and point out two major issues: high variance in their standard deviation, and high correlation between different dimensions. We also propose a new network regularization method, isotropic batch normalization (IsoBN) to address the issues, towards learning more isotropic representations in fine-tuning by dynamically penalizing dominating principal components. This simple yet effective fine-tuning method yields about 1.0 absolute increment on the average of seven NLU tasks.Comment: AAAI 202

Creative users, social networking, and new models of publishing

This paper reviews the changing landscape of the publishing industry, which is being reshaped by dynamics of user co-creation, social networking and open licencing. It briefly touches on possible research themes associated with disruptive changes in the world's oldest media/creative industry, particularly under the umbrella of 'Cultural Science'. Two new models of publishing are discussed: literary self-publishing in China and open innovations in academic publishing. It argues that evolution in the publishing industry goes beyond 'digital publishing' towards 'new publishing in a digital world', demanding new models serving population-wide creativity and open knowledge communication